1. Field of the Invention
The present invention relates to an image processing apparatus that captures an image of a target object by use of a solid state image sensor, a control method thereof, and also relates to a computer-readable storage medium that stores therein a computer-readable program that makes a computer execute the control method.
2. Description of the Related Art
In recent years, image pick up apparatuses such as digital cameras, which are capable of recording and/or reproducing images captured by a solid state image sensor such as CCD (hereinafter may be referred to just as an “image sensor”) via a storage medium such as a memory card equipped with solid state memory elements, have been developed actively and become widely used. In this kind of image pick up apparatus, improvements in resolution and operational speed that are concerned with capturing ability of still images and motion images have been demanded. In connection with this, a drive signal frequency for driving an image sensor, and also a drive signal frequency for driving an analog signal processing circuit, an A/D converter and a subsequent digital signal processing circuit have also been made much faster nowadays.
Further, in addition to an improvement of an image quality such as high quality and high fidelity, an easy function (handiness) of enabling a failure-proof image capturing operation in various image capturing occasions has also been demanded to image pick up apparatuses. For example, in order to properly capture the image of a target object moving at a high speed, just as a case of capturing an image of a sport scene or the like, or in order to suppress a so-called camera shake just in an occasion of taking a picture in a low-illuminated dark room or the like, there has been a remarkable progress in making a shutter speed faster. Furthermore, in order to enable an image capturing operation in a place like a museum, an aquarium and so forth in which an image capturing by use of a stroboscope is prohibited, image sensors with higher sensitivity have also been demanded. Now an explanation about a conventional image pick up apparatus is given below.
FIG. 8 is a block diagram showing a schematic configuration of an image capturing unit of a conventional image pick up apparatus such as a digital camera.
In FIG. 8, an image sensor (hereinafter CCD) 501 coverts an optical image of a target object to an image signal, which is an electric signal. An image capturing circuit 502 processes the image signal output from the CCD 501. An A/D converter 503 converts the thus processed analog image signal to a digital image signal. A rectangular frame denoted by a reference numeral 500 represents an analog signal processing area. A digital signal processing unit 504 carries out various signal processing such as storing a digital-converted image signal in a storage medium and displaying an image based on by the digital-converted image signal on a liquid crystal display monitor. A system control unit 509 contains therein a CPU that totally controls the operations of the digital camera.
An oscillation circuit 505 (OSC1) feeds an operation clock for a timing generator 506. An oscillation circuit 508 (OSC2) feeds an operation clock for the system control unit 509. The timing generator 506 feeds an operation clock (TGCLK) to a synchronous signal generator 507 (SSG). The synchronous signal generator 507 (SSG) counts the number of operation clock up to a predetermined number and generates a horizontal synchronous signal (HD) and a vertical synchronous signal (VD) so as to feed them to the timing generator 506 thereafter.
The timing generator 506 feeds various drive signals (H and V drive pulses) to the CCD 501 in synchronization with the horizontal and vertical synchronous signals. The timing generator 506 further feeds sampling clock signals to the image capturing circuit 502, the A/D converter 503, and also to the digital signal processing unit 504 respectively. The system control unit 509 instructs the synchronous signal generator 507 to generate horizontal and vertical synchronous signals and to set their cycles, and also controls the operation of the digital signal processing unit 504.
FIG. 9 is a view showing a schematic configuration of the CCD 501.
In FIG. 9, the CCD 501 is provided with a plurality of photoelectric conversion elements 511, a vertical transfer CCD (VCCD) array 512 and a horizontal transfer CCD (HCCD) array 513, wherein both the VCCD array 512 and HCCD array 513 are respectively also provided with a plurality of photoelectric conversion elements. Within the photoelectric conversion elements 511, photoelectric conversion elements 511a arranged in the leftmost side are shaded, while other photoelectric conversion elements 511b are arranged within a non-shaded active pixel area. Each of the photoelectric conversion elements 511 and corresponding one in the VCCD array 512 configure a paired unit, and an image sensing area is formed by providing a plurality of these paired units in a two-dimensional plane, wherein the light beams from a target object are converted into electric charges so as to form an image. The VCCD array 512 sequentially transfers the electric charges to the HCCD array 513, and the HCCD array 513 further transfers the thus transferred electric charges in the horizontal direction.
Describing this in detail, each the electric charges generated in the photoelectric conversion elements 511 is first transferred to the corresponding paired element in the VCCD array 512, and then the electric charges transferred to the VCCD array 512 are transferred to the HCCD 513 in the vertical direction sequentially, taking each of horizontal lines as one unit (in the line-by-line base). The electric charges thus transferred in the vertical direction are further transferred in the horizontal direction by the HCCD array 513, and finally converted to electric voltages to be output by an amplifier 514, which converts the electric charges to the electric voltages. Actually the VCCD array 512, the HCCD array 513 and the photoelectric conversion elements 511a and 511b, which configure the CCD 501 altogether, could be many more than the number of pieces described in FIG. 9. For example, although the photoelectric conversion elements 511a are illustrated as one vertical line in the leftmost position, there are, in fact, many more vertical lines in reality.
FIG. 10 is a block diagram showing a detailed configuration of the image capturing circuit 502.
In FIG. 10, the image capturing circuit 502 is provided with a correlated double sampling (CDS) circuit 600, an amplifier 601 and a clamp circuit 602. Generally, the CDS circuit 600 is provided in the down stream side of the CCD 501 so as to remove the reset noise component generated during the charge-transfer in the CCD 501. The output from the CCD 501 is composed of a field-through section and a motion image signal section. The field-through section becomes a base for the signal level of each pixel within one horizontal transfer cycle. The motion image signal section outputs a motion image signal in proportion to the amount of exposure.
The CDS circuit 600 is a noise canceling circuit that gains a difference between the level of the field-through section and that of the motion image signal section within the output signal of the CCD 501, and removes the correlated noise components within one pixel cycle from the motion image signal. The amplifier 601 amplifies the motion image signal output from the CDS circuit 600 to a predetermined signal level in accordance with the input range of the A/D converter 503 provided in its down stream side, and feeds it to the clamp circuit 602. The clamp circuit 602 adjusts a DC voltage level in such a manner that the each of the electric charges outputted from the pixels resided in the shaded part of the CCD 501 becomes a predetermined black reference value. The period during which the electric charge is outputted from each of the pixels in the shaded part is called an optical black (OB) period.
FIG. 11 is a timing chart showing the main signals for driving the above-noted image pick up apparatus.
In FIG. 11, the frequency of an operation clock of the oscillation circuit 505 in FIG. 8 is set to 33.75 MHz, and that of the oscillation circuit 508 in FIG. 8 is set to 27 MHz. The frequency of an operation clock for each pixel output from the CCD 501 is determined in accordance with the CCD drive signal generated by the timing generator 506, which is generated based on 33.75 MHz that is same as the frequency of the operation clock of the oscillation circuit 505. In other words, one pixel period (=one pixel cycle) of the output signal of the CCD 501 in this case is 29.6 ns (=1/33.75 MHz), within which the above-noted field-through section and the image motion signal section are included.
Further, in the timing generator 506, an S/H pulse (SH1) that samples and holds the signal level of the field-through section and an S/H pulse (SH2) that samples and holds the signal level of the motion image signal section are generated for each pixel so as to synchronize with the CCD drive signal.
The following apparatus has been proposed as a related art of the above-mentioned technical field (for example, Japanese Patent Laid-Open Publication (KOKAI) No. 2001-285726). In this Publication, a technique is proposed to suppress undesired beat noise generated when the frequency diffusion unit is used for processing of analog signals of an image pick up apparatus.
However, there has been such a problem in the above-noted conventional technique. That is, the speeding up of the drive frequency of an image signal can be a great factor to cause a deterioration of an signal-to-noise (S/N) ratio. Specially, within an image pick up apparatus such as a digital camera which is driven by a plurality of operation clock signals and both analog signals and digital signals are used therein, undesired clock signals leaked into analog image signals are likely to occur. The undesired clock signals leaked into image signals are superimposed on a generated image at equal pitches as an interfering clock noise.
Since the interfering clock noise is generated at equal pitches, even if its noise level is smaller than a heat-noise-type random noise of the CCD or the circuit, its conspicuousness has been a great problem. In addition, the above-noted deterioration of a signal-to-noise ratio of an image signal can be more conspicuous, when the sensitive condition of the image pick up apparatus is set to a high level, and the amplification level of an image signal in the image capture circuit is set to a large level.
For example, with the configuration as shown in FIG. 8, in accordance with the speeding up of the system operation and of the drive frequency of images signals, the S/H pulses (SH1 and SH2) and the sampling clock (ADCLK) for the A/D converter 503 have also been speeded up. Due to this, it has been made more difficult to remove the undesired clock noise component (system clock component) leaked into an image signal in the analog signal processing area 500 by a timing adjustment.
Here, a case in which the output of the CCD 501 is sampled by use of the S/H pulses (SH1, SH2) whose pixel-clock frequency is 33.75 MHz and a sampling clock (ADCLK) of the A/D converter 503 is now described hereinafter. When the system clock component (27 MHz) is leaked into image signals in the analog signal processing area 500, the periodic noise of 6.75 MHz (=33.75 MHz−27 MHz) that is a differential value of the frequency component remains in the image data after the sampling operation is completed.
The periodic noise of 6.75 MHz is an equal-pitch noise which has a frequency of one-fifth of 33.75 MHz that is a drive pulse of the CCD 501, and which has 5-pixel cycle, wherein one horizontal cycle is composed of 5 pixels. The feature of an equal-pitch noise generated by interfering pulses can be readily observed, although depending on its pitch size, in comparison with a heat-noise type random noise contained in the image sensing circuit 502 or in the CCD 501, thus it will be quite noticeable.
In the case of the CCD 501, a general configuration (timing chart) of one horizontal cycle is as shown in FIG. 12. In other words, one horizontal cycle is composed of a blanking period in which the transferring drive pulses H1 and H2 for the HCCD array 513 are stopped, and a pixel reading period during which the transferring pulses H1 and H2 for the HCCD array 513 are activated (OB pixel period+active pixel period).
Visual impression of an equal-pitch noise that is one-dimensionally superimposed on an image signal varies depending on the number of pixel clocks composing one horizontal cycle, with respect to a two-dimensional image expanded in the horizontal and vertical directions in the CCD area sensor.
For example, in the case of an equal-pitch noise of 5-pixel cycle, the noise pattern formed during that period contains 5 variations that are based on a coset of 5 in accordance with the number of pixel clocks composing one horizontal cycle as shown in FIGS. 13A to 13C, wherein symbols “N” in these figures correspond to integers. As can be observed from FIGS. 13A to 13E, although there is no change in noise pitches in the horizontal direction, since the angle of the noise pattern varies on an expanded two-dimensional image, it has been confirmed that the level of its noticeability varies.
FIG. 14 is a timing chart showing the timing for image capturing operation when taking a motion picture by the image pick up apparatus.
In FIG. 14, Tfc denotes a vertical cycle and is a period during which one field signal (one scene) is output when driving the CCD 501. The vertical cycle Tfc is set to an image capturing rate (for example, 60 scenes per second) that is suitable for smoothly capturing a motion image of a target object. VD denotes a vertical synchronous signal, and HD denotes a horizontal synchronous signal.
Exposure is controlled by mechanically opening and closing a mechanical diaphragm (not shown), adding electronic shutter pulses to the substrate's potential of the CCD 501 for a predetermined term Tfe, to thereby reset the electric charges of pixels in the direction of the substrate. The time period Tsc, which is a time duration from the moment of a completion of the reset of the pixel charges by the electronic shutter until the time on which the pixel charges of the photoelectric conversion elements 511 are read out by the VCCD 512, will be regarded as being a time period for exposure and accumulation in a filed signal of the CCD 501.
FIGS. 15A to 15C are views that depict an example of a noise pattern when the above-noted equal-pitch noise of the 5-pixel cycle that is superimposed on a one-dimensional image signal is expanded on a two-dimensional image of each field configuring a motion image at the time of reading out the motion image in the image pick up apparatus using a CCD area sensor.
FIGS. 15A to 15C represent a noise pattern in the case that three fields of the noise pattern shown in FIG. 13B are combined for one frame.
FIG. 15A is a view showing a state of noise patterns superimposed one another over the entire period including the blanking periods where no image is displayed. FIG. 15B is a view showing the state of a noise pattern in the image display area of the Kth field of the field signals that vary in accordance with a lapse of time. FIG. 15C is a view showing the state of a noise pattern in which when the Kth field image is displayed, the noise patterns of the (K−1)th and (K−2)th field images are superimposed on the Kth field image, due to the responsibility of the display device and/or an afterimage effect.
As shown in FIGS. 15A to 15C, since one horizontal cycle is constant for each of the field signals, even though the position of the noise patterns expanded two-dimensionally on the field image shifts for each field, the angle of each of the noise patterns does not change. Due to this fact, noise patterns are observed on motion images composed of these field images, as if they scroll upward or sideward in accordance with a lapse of time, keeping their periodic features that can be observed at the same angle and an equal pitch. This makes the noise patterns conspicuous in some cases.
If the shift value of the position of the noise patterns between respective fields can be controlled well, a motion image in which a noise pattern is not so conspicuous can be configured to some extent by making the scrolling speed of the noise pattern faster.
However, as shown in FIG. 8, there is a configuration in which the oscillation circuit 505 for the operation clock (33.75 MHz) and the oscillation circuit 508 for the operation clock (27 MHz) are used. In other words, there are many cases in which multiple oscillation circuits are independently used corresponding to their individual frequencies. In such a configuration (so-called free-run configuration) in which the oscillation circuits are independent each other in accordance with their individual frequencies, as the phase of a noise pattern in each field moves depending on the precision and temperature drift of each oscillation circuit, a shift value of the position of the noise patterns between each field can be hardly controlled.
For example, it is possible to generate an operation clock of the oscillation circuit 505 (33.75 MHz) from the operation clock of the oscillation circuit 508 (27 MHz) by use of a Phase Locked Loop (PLL) circuit or the like. In this case, as the noise pattern and a phase relation between respective fields can be precisely determined, the shift value of the position of the noise pattern between respective fields may be controlled.
In this case, however, the operation clock (27 MHz) of the oscillation circuit 508 must be supplied by way of the PLL circuit to the timing generator 506 provided in the nearby area of the analog signal processing area 500 that is quite sensitive to noise, and due to this, there is such a great risk that the system operation clock (27 MHz) is likely to be leaked into analog image signals.
According to the above-noted Japanese Patent Laid-Open Publication (KOKAI) No. 2001-285726, a technique is disclosed that a beat noise that responds to a cycle of the frequency diffusion attributable to a frequency diffusing unit is superimposed on an image signal, whereby the beat noise is reduced by resetting the phase of the frequency diffusing unit at random timings in the horizontal transfer blanking period of the image sensor.
However, in case of a image pick up apparatus such as a digital camera that is not provided with the frequency diffusing unit as shown in FIG. 8, there has been a problem that undesired periodic noise generated in a unit other than the frequency diffusing unit cannot be reduced by the method disclosed in the above noted Laid Open document.